Weighted Point Cloud Augmentation for Neural Network Training Data Class-Imbalance

Recent developments in the field of deep learning for 3D data have demonstrated promising potential for end-to-end learning directly from point clouds. However, many real-world point clouds contain a large class im-balance due to the natural class im-balance observed in nature. For example, a 3D scan of an urban environment will consist mostly of road and facade, whereas other objects such as poles will be under-represented. In this paper we address this issue by employing a weighted augmentation to increase classes that contain fewer points. By mitigating the class im-balance present in the data we demonstrate that a standard PointNet++ deep neural network can achieve higher performance at inference on validation data. This was observed as an increase of F1 score of 19% and 25% on two test benchmark datasets; ScanNet and Semantic3D respectively where no class im-balance pre-processing had been performed. Our networks performed better on both highly-represented and under-represented classes, which indicates that the network is learning more robust and meaningful features when the loss function is not overly exposed to only a few classes.Comment: 7 pages, 6 figures, submitted for ISPRS Geospatial Week conference 201

Open ML Training Data For Visual Tagging Of Construction-specific Objects (ConTag)

ConTag has generated open datasets for visual machine learning (ML) specific to the construction industry. ML technology has enabled a revolutionary leap in many digital economies generating growth in activity and business mainly for the ITC sector. Part of the growth is generated through sharing of IP, knowledge, tools and datasets. We want to adopt this approach for the digital construction sector. ConTag provides visual and 3D training datasets for training deep neural networks (DNNs) and provides weights for pre-trained networks. The research output is to support visual tagging of assets from reality capture data. Such automatically generated semantic information can be used to generate or populate digital twins in the example scenarios. The first dataset is a collection of fire safety equipment typically found in indoor environments. The dataset contains the classified images, per-pixel label images and bounding box data for object detection. The second dataset is a synthetic 3D point cloud of an outdoor urban street scenario. The dataset contains the point cloud data and per-point label data. We expect this shared and open datasets to kick-start further ML developments in both academia and industry. It is intended as a seed point for collaborative research

Potential of Consumer-Grade Cameras and Photogrammetric Guidelines for Subsurface Utility Mapping

The poor documentation of subsurface utility data is a common problem in many cities, exposing field engineers to risks of utility strike. This paper investigates the use of consumer-grade cameras to improve operational efficiency on construction sites and explores different imaging networks to optimize photogrammetric processing for low-cost subsurface utility surveys. Results from the first part of the study demonstrated the potential of consumer-grade cameras as a photogrammetric utility data acquisition tool. However, statistical insights from the photogrammetric calibration show that caution needs to be taken about the camera types particularly for lens calibration. Results from the second part of the study were recommended as easy-to-understand guidelines for image acquisition at trenches and supported the planning of photogrammetric measurements in the field

A review on deep learning techniques for 3D sensed data classification

Over the past decade deep learning has driven progress in 2D image understanding. Despite these advancements, techniques for automatic 3D sensed data understanding, such as point clouds, is comparatively immature. However, with a range of important applications from indoor robotics navigation to national scale remote sensing there is a high demand for algorithms that can learn to automatically understand and classify 3D sensed data. In this paper we review the current state-of-the-art deep learning architectures for processing unstructured Euclidean data. We begin by addressing the background concepts and traditional methodologies. We review the current main approaches including; RGB-D, multi-view, volumetric and fully end-to-end architecture designs. Datasets for each category are documented and explained. Finally, we give a detailed discussion about the future of deep learning for 3D sensed data, using literature to justify the areas where future research would be most valuable.Comment: 25 pages, 9 figures. Review pape

Vertical integration and foreclosure: evidence from production network data

This paper studies the prevalence of vertical market foreclosure using a novel dataset on U.S. and international buyer-seller relationships, and across a large range of industries. We find that relationships are more likely to break when suppliers vertically integrate with one of the buyers' competitors than when they vertically integrate with an unrelated firm. This relationship holds for both domestic and cross-border mergers, and for domestic and international relationships. It also holds when instrumenting mergers using exogenous downward pressure on the supplier's stock prices, suggesting that reverse causality is unlikely to explain the result. In contrast, the relationship vanishes when using rumoured or announced but not completed integration events. Firms experience a substantial drop in sales when one of their suppliers integrates with one of their competitors. This sales drop is mitigated if the firm has alternative suppliers in place

SynthCity: A large scale synthetic point cloud

With deep learning becoming a more prominent approach for automatic classification of three-dimensional point cloud data, a key bottleneck is the amount of high quality training data, especially when compared to that available for two-dimensional images. One potential solution is the use of synthetic data for pre-training networks, however the ability for models to generalise from synthetic data to real world data has been poorly studied for point clouds. Despite this, a huge wealth of 3D virtual environments exist which, if proved effective can be exploited. We therefore argue that research in this domain would be of significant use. In this paper we present SynthCity an open dataset to help aid research. SynthCity is a 367.9M point synthetic full colour Mobile Laser Scanning point cloud. Every point is assigned a label from one of nine categories. We generate our point cloud in a typical Urban/Suburban environment using the Blensor plugin for Blender.Comment: 6 pages, 4 figures, dataset white pape

Finding Your (3D) Center: 3D Object Detection Using a Learned Loss

Massive semantically labeled datasets are readily available for 2D images, however, are much harder to achieve for 3D scenes. Objects in 3D repositories like ShapeNet are labeled, but regrettably only in isolation, so without context. 3D scenes can be acquired by range scanners on city-level scale, but much fewer with semantic labels. Addressing this disparity, we introduce a new optimization procedure, which allows training for 3D detection with raw 3D scans while using as little as 5% of the object labels and still achieve comparable performance. Our optimization uses two networks. A scene network maps an entire 3D scene to a set of 3D object centers. As we assume the scene not to be labeled by centers, no classic loss, such as Chamfer can be used to train it. Instead, we use another network to emulate the loss. This loss network is trained on a small labeled subset and maps a non centered 3D object in the presence of distractions to its own center. This function is very similar - and hence can be used instead of - the gradient the supervised loss would provide. Our evaluation documents competitive fidelity at a much lower level of supervision, respectively higher quality at comparable supervision. Supplementary material can be found at: https://dgriffiths3.github.io.Comment: 19 pages, 8 figures, Accepted ECCV 202

Curiosity-driven 3D Object Detection Without Labels

In this paper we set out to solve the task of 6-DOF 3D object detection from 2D images, where the only supervision is a geometric representation of the objects we aim to find. In doing so, we remove the need for 6-DOF labels (i.e., position, orientation etc.), allowing our network to be trained on unlabeled images in a self-supervised manner. We achieve this through a neural network which learns an explicit scene parameterization which is subsequently passed into a differentiable renderer. We analyze why analysis-by-synthesis-like losses for supervision of 3D scene structure using differentiable rendering is not practical, as it almost always gets stuck in local minima of visual ambiguities. This can be overcome by a novel form of training, where an additional network is employed to steer the optimization itself to explore the entire parameter space i.e., to be curious, and hence, to resolve those ambiguities and find workable minima.Comment: 19 pages, 17 figure

Improving Microbiological Safety and Quality Characteristics of Wheat and Barley by High Voltage Atmospheric Cold Plasma Closed Processing

Contamination of cereal grains as a key global food resource with insects or microorganisms is a persistent concern for the grain industry due to irreversible damage to quality and safety characteristics and economic losses. Atmospheric cold plasma presents an alternative to conventional grain decontamination methods owing to the high antimicrobial potential of reactive species generated during the treatment, but effects against product specific microflora are required to understand how to optimally develop this approach for grains. This work investigated the influence of ACP processing parameters for both cereal grain decontamination and grain quality as important criteria for grain or seed use. A high voltage (HV) (80 kV) dielectric barrier discharge (DBD) closed system was used to assess the potential for control of native microflora and pathogenic bacterial and fungal challenge microorganisms, in tandem with effects on grain functional properties. Response surface modelling of experimental data probed the key factors in relation to microbial control and seed germination promotion. The maximal reductions of barley background microbiota were 2.4 and 2.1 log10 CFU/g and of wheat - 1.5 and 2.5 log10 CFU/g for bacteria and fungi, respectively, which required direct treatment for 20 min followed by a 24 h sealed post-treatment retention time. In the case of challenge organisms inoculated on barley grains, the highest resistance was observed for Bacillus atrophaeus endospores, which, regardless of retention time, were maximally reduced by 2.4 log10 CFU/g after 20 min of direct treatment. The efficacy of the plasma treatment against selected microorganisms decreased in the following order: E. coli \u3e P. verrucosum (spores) \u3e B. atrophaeus (vegetative cells) \u3e B. atrophaeus (endospores). The challenge microorganisms were more susceptible to ACP treatment than naturally present background microbiota. No major effect of short term plasma treatment on the retention of quality parameters was observed. Germination percentage measured after 7 days cultivation was similar for samples treated for up to 5 min, but this was decreased after 20 min of direct treatment. Overall, ACP proved effective for cereal grain decontamination, but it is noted that the diverse native micro-flora may pose greater resistance to the closed, surface decontamination approach than the individual fungal or bacterial challenges, which warrants investigation of grain microbiome responses to ACP